Exploring Temporal Vagueness with Mechanical Turk

This paper proposes schematic changes to the TempEval framework that target the temporal vagueness problem. Specifically, two elements of vagueness are singled out for special treatment: vague time expressions, and explicit/implicit temporal modification of events. As proof of concept, an annotation experiment on explicit/implicit modification is conducted on Amazon’s Mechanical Turk. Results show that the quality of a considerable segment of the annotation is comparable to annotation obtained in the traditional doubleblind setting, only with higher coverage. This approach offers additional flexibility in how the temporal annotation data can be used.